On the differential benchmarking of promotional efficiency with machine learning modeling (I): Principles and statistical comparison

Sales promotions have become in recent years a paramount issue in the marketing strategies of many companies, and they have even more relevance in the present economic situation. Currently, the empirical models, aimed at assessing consumers behavior in response to certain sales promotions activities such as temporary price reductions, are receiving growing attention in this relevant research field, due to two reasons mainly: (1) the complexity of the interactions among the different elements incorporated inside promotions campaigns attracts growing attention; and (2) the increased availability of electronic records on sales history. Hence, it will become important that the performance description and comparison among all available machine learning promotion models, as well as their design parameters selection, will be performed using a robust and statistically rigorous procedure, while keeping functionality and usefulness. In this paper, we first propose a simple nonparametric statistical tool, based on the paired bootstrap resampling, to allow an operative result comparison among different learning-from-samples promotional models. Secondly, we use the bootstrap statistical description to evaluate the models in terms of average and scatter measurements, for a more complete efficiency characterization of the promotional sales models. These statistical characterizations allow us to readily work with the distribution of the actual risk, in order to avoid overoptimistic performance evaluation in the machine learning based models. We also present the analysis performed to determinate whether the figure of merit has a significant impact on final result, together with an in depth design parameter selection to optimize final results during the promotion evaluation using statistical learning techniques. No significant difference was obtained in terms of figure of merit choice, and Mean Absolute Error was selected for performance measurement. As a summary, the applied technique allows clarifying the design of the promotional sales models for a real database (milk category), according to the influence of the figure of merit used for design parameters selection, showing the robustness of the machine learning techniques in this setting. Results obtained in this paper will be subsequently applied, and presented in the companion paper, devoted to a more detailed quality analysis, to evaluate four well-known machine learning algorithms in real databases for two categories with different promotional behavior. 2012 Elsevier Ltd. All rights reserved.